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SUMMARY 

OBJECTIVE 

Determination  of  the  first  two  moments  of  the  ALE  output  detector  statistics 
for  sinusoids  in  white  noise. 

RESULTS 

Moments  were  determined  for  the  output,  short-term  integrated  output,  and  long- 
term integrated  output  power  spectra. 

RECOMMENDATIONS 

These  results  may  be  implemented  on  a computer  to  calculate  ROC  curves  for  the 
ALE  output  detector  and  thus  aid  in  the  determination  of  parameters  such  as  p,  filter 
length,  and  FFT  resolution. 
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INTRODUCTION 


This  paper  consists,  basically,  of  a computation  of  the  first  two  moments  of  the 
output  distribution  of  the  adaptive  line  enhancer  (ALE)  detector  (figures  1 and  2).  Its 
primary  motivation  is  to  provide  a means  of  determining  the  detection  characteristics  of 
the  integrated  spectral  output. 

Previous  results  [ I ] have  relied  on  assumptions  regarding  the  output  statistics  them- 
selves; i.e.,  that  they  be  chi-square  distributions.  Although  such  an  assumption  greatly 
simplifies  the  analysis,  it  introduces  an  artificial  relationship  between  the  first  two  moments, 
which  prevents  a rigorous  application  even  in  the  limiting  Gaussian  case  typical  of  post- 
detection integration.  In  the  present  work  we  shall  make  only  two  assumptions;  that  the  in- 
put to  the  detector  be  a sinusoid  in  white  Gaussian  noise,  and  that  the  discrete  Fourier  trans- 
form (DFT)  length  and  ALE  filter  length  be  considered  smaller  than  r,  the  adaptive  time 
constant  of  the  ALE  filter. 

We  then  compute  the  moments  of  the  ALE  detector  for  three  cases:  (a)  a single  power 
spectrum;  (b)  a short  term  average  of  the  output  (integration  time  < r);  and  (c)  long-term 
average  (integration  time  > r).  The  distinction  between  the  two  types  of  averaging  is  related 
to  the  behavior  of  the  converged  ALE  filter  which  is  nearly  constant  over  periods  of  time  less 
than  r [2]. 

The  above  results  are  independent  of  any  assumptions  on  the  output  statistics.  If  a 
model  for  the  distribution  is  known,  for  example  empirically,  the  receiver  operating  charac- 
teristic (ROC)  of  the  detector  may  be  computed.  In  particular,  for  second  order  distributions, 
the  ROC  curves  are  determined  by  the  first  two  moments  as  follows.  Let  pg  and  p j be  the 

normalized  probability  densities  (mean  zero  and  variance  1)  corresponding  to  the  output 
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under  the  hypotheses  signal,  absent  and  signal  present,  respectively.  Also,  let  pg,  p j , <Jq  and 
a“  be  their  means  and  variances.  Then  the  probabilities  of  detection  and  false  alarm  are  given 
by 


OO 


°0 


PD  = 


/ 

t->m 

°\ 


p\ 


(1) 


For  example,  in  the  case  of  a Gaussian  distribution  (which  occurs  for  sufficently 
large  integration  times)  for  a given  false  alarm  rate,  the  threshold,  a,  may  be  found  in 
standard  tables.  Then 


T - 


°0 

which  implies 


^0 

— = a , 


(2) 


T-Mi  QgQ  + MQ-Mi 
a,  »1 


(3) 


and  Py  is  also  determined  from  tables. 


MOMENTS  OF  INTEGRATED  OUTPUT 


Let  y be  the  output  of  the  detector  pictured  in  figure  1 given  by 

y = |7(w  * x)|2  (4) 

where  w and  x are  the  weight  vector  and  input  vector  of  the  ALE,  * denotes  convolution, 
and  ? indicates  a K point  discrete  Fourier  transform  (DFT)  at  frequency  co.  For  a sinusoid 
input. 


Xg(k)  = A cos  co(k  - S)  + n(k)  . (5) 

The  converged  weight  vector  may  be  written  [ 1,2,3) 

Wg(k)  = B cos  (ooC  + </>)  + ng(k)  , (6) 

where  n(k)  is  a Gaussian  white  noise  process  and  n'(k)  is  an  independent  stationary  Gaussian 
vector  process  with  a correlation  time  of  the  order  r.  The  individual  components  n^  are 
independent  identically  distributed  random  variables. 


z(k) 


Figure  1.  Block  diagram  of  the  ALE  output  detector. 

The  first  moment  of  y was  computed  in  [ 1 1 . The  computation  of  the  second 
moment  is  straightforward,  although  extremely  tedious,  and  has  been  relegated  to  the 
Appendix.  We  therefore  address  ourselves  here  to  a discussion  of  the  effects  of  averaging. 
To  do  this,  it  is  convenient  to  separate  the  effects  of  x and  w through  the  use  of  conditional 
expectations. 
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Let  E(  *|w)  denote  conditional  expectation,  given  w.  Define 
v = E(y  I w) 

= E(y)  . (7> 

Then  v is  a random  variable  dependent  only  on  w and 


E(v)  = E(y)  = fiy  . (8) 

It  follows  that  E(y  - v)  = 0,  and 

E^v(y-v))  =E  Je  (v(y-v)|w)|  = E [vE(y — v|w)|  =0  . 

Thus  we  may  write 

y = (y-v)  + v 
and 

varty)  = var(y-v)  + var(v)  , (9) 

where  var  indicates  variance. 


SHORT-TERM  AVERAGE 

Let  y(m)  denote  the  output  due  to  the  2mth  DFT.^  The  output  is  chosen  every  other 
DPT  because,  as  a result  of  the  convolution  in  (4),  it  is  independent  with  respect  to  x only 
every  other  DFT  (every  K + L points  for  K < L).  The  effects  of  averaging  every  DFT  or  of 
overlapping  DFT’s  may  be  computed  after  the  manner  of  Appendix  B,  but  in  most  cases  of 
practical  interest  will  be  inconsequential  (cf(21)). 

Suppose  that  w(k  + mK)  is  essentially  constant  for  k < K,  m = 1 M and  MK  < r. 

Although  this  is  usually  the  case,  it  is  only  necessary  to  assume  that  K < r.  A more  accurate 
treatment  appears  in  Appendix  B.  Define  the  short-term  average  variable  z by 

M 

z=si  X y(m) 

m=  1 
M 

= jjj  ^ (y(m>- v)  + v , (10) 

m=l 


^actually  m • max  (2K,  K + L)/K 
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which  follows  since  v depends  only  on  w and  hence  is  independent  of  m.  Let  us  determine 
the  variance  of  the  first  term  in  (10): 


E (n-  S (y(m)  - v)2)  = 2 S E(y(m)  - v)(y(r)  - v) 

v m,r 


_ — E 
M2 


^ E ((y(m)  - v)(y(r)  - v)|w) 


m,r 


___  E 
M2 


^ 6m,rE  ((y(m)-v)2|w) 


Lm,r 


=ME(y-v>2  • 


do 


Since  the  first  term  of  ( 10)  has  zero  mean,  and  since  E(v(y(m)-v))  = 0,  equations ( 10) 
and  (11)  yield 


var  (z)  = jjj  var  (y  - v)  + var  (v)  . 


From  (9)  and  ( 1 2) 

var  (z)  = var  (y)  + (j^-  l)  var(y-v)  . 


(12) 


(13) 


VARIANCE  OFy-v 

We  now  express  the  second  term  in  (13)  in  a form  suitable  for  computation.  Since 
E(y-v)  = 0, 


var(y  - v)  = E(y  - v)2 

= E Je  ((y  - v)2  lw)| 

= E(varx(r)l  (14) 

where  r is  equal  to  y,  considered  as  a random  variable  in  x with  w as  a parameter.  Note  that 
the  “mean”  of  r is  E(y|w)  = v,  and  both  its  “mean”  and  “variance”  are  functions  of  the 
random  variable  w.  The  subscript  x is  included  to  avoid  confusion  with  expectation  taken 
with  respect  to  the  entire  sample  space. 

From  (4),  we  see  that  for  fixed  w,  7(w  * x)  is  complex  Gaussian,  and  hence  r is 
chi-square  (possibly  noncentral)  of  two  degrees  of  freedom.  Let  the  mean-squared  and  one- 
half  the  variance  of  the  complex  Gaussian  process  ?(w*x)  be  given  by 
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05) 


t}‘=  |Ex7(w*x)|2 

Xr:  = T(Exl7(w*x)|2-Tj“)  . 

2 2 

Then  the  mean  and  second  moment  of  r are  (31  (the  noncentrality  parameter  is  tj~ /X“) 


meanY  r = v = 2 X“  + t)~ 

A I I 

varv  r = 4 X^  + 4 X“  . (^) 

^ r r r 

It  follows  from  (16) 

">  4 ( | 

varx  r = v“  - r?r  . v 1 '' 

Equations  ( 1 4)  and  ( 1 7)  yield 

E(var  (y  - v)l  = E(v~)  - E(t)^)  . 

Also,  from  (9) 

E(var  (y  - v)]  = E(y“)  - E(v~)  . U9) 

Adding  these  two  expressions,  we  have 

E[var(y  - v)]  =4;  E(y~) -4  E(ij^)  • 

In  this  manner,  we  have  removed  the  variable  v from  our  calculations.  The  substitu- 
tion of  (20)  in  ( 13)  yields 


var(z)=E(y2)(^+^)  -My  + E(r?J)  (4"2m)  • 


(21) 


j 2 

To  evaluate  this  expression  we  need  to  know  the  first  two  moments  of  y (E(y-)  and  Py ) and 
E(rj^).  The  first  moment  is  available  in  [ 1 ] ,t  the  second  is  calculated  in  Appendix  A,  and  we 
proceed  to  determine  E(rj^). 


^ In  the  notation  of  [ 1 ] , E(y)  = 0“  + 0~  + u"  + v2  • 
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CALCULATION  OF  E(t}J) 

Let  L be  the  length  of  the  ALE.  In  order  to  simplify  calculations,  it  is  assumed  that 
the  analysis  frequency  cu  is  bin-centered,  both  with  respect  to  the  DFT  and  ALE;  i.e., 

WK  = multiple  of  2ir  and  cuL  = multiple  of  2n.  This  assumption  only  affects  terms  of  lower 
order  in  K and  L (cf  Appendix  A),  and  hence  will  not  change  the  final  result. 

For  w fixed  the  mean  of?(w*x)  is  given  by 

L K 

Ex  (j(w*x))  =EX  ]>  X wgxg(k0  + n)e-lton 
8= 1 n=l 

L K 

= ^ ^ Wg  A cos  ^co(n  -2)  + e-'1011 

8=1  n=l 

L K 

= ^ Wge-'w^  ^ A cos  ^cu(n  - 2)  + <^>)e'* 

8=1  n=l 

L K-2 

= ^ Wj,  ^ A cos  (cun' + </>)  e-^11  . 

8=  1 n'=l-8 

Since  K is  a multiple  of  the  period  of  cos  (cun'  + </>),  the  second  sum  is  independent  of  2,  and 
Ex(3(w*x))  = ?l(w)  7j^(A  cos  (cun  + 0))  . (22) 


Thus,  from  definitions  ( 1 5)  and  (22) 

(23) 

'y 

We  now  note  that  I7ijw)h  considered  as  a function  of  w is  a chi-square  random 
variable  with  two  degrees  of  freedom.  Let 

72  = F.(n'g ) . (24) 

Then  expressions  (15)  and  ( 1 6)  apply  with 
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and 


2 L2B2 


TJw=  4 


Thus 


EU^Cw)!^  = second  moment 


_fiLi^+8L^2B2.L4B4 


=84'-  8 

Combining  this  with  (23)  we  have 


16  ‘ 


(25) 


r,  4,.A4K4  /m2  2 , , U1  2 . L4  B4\ 

E(V T6-\2L"^+L'B"^  + -7r//  ' 


(26) 


LONG-TERM  AVERAGE 

Consider  the  average  output  over  a time  longer  thanr 
NM 


1 


z NM  ^ 
G=1 


(27) 


This  may  be  rewritten,  for  MK  < r. 


N / M \ 

= N S ( M S y(sM  + «)j 
s=l  \ «=I  / 


N 


= R 2 z(s)  • 

s=  1 


(28) 


The  random  variables  y are  a function  of  x and  w.  Since  the  same  x(k)  do  not  appear  in 
y(k  | ) and  y(k2)  for  k | =£  k2.  and  since  the  time  dependence  of  w(k)  is  exponential  with  a 

correlation  time  r,  z(s)  has  a time  dependence  of  2 r (w  is  squared  in  equation  (4)) 


E ^z(S|)z(s2)^  ~ E(z2)  e S2'^T 


(29) 
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I 


Consequently,  if  N = T/r  » 1 , where  T is  the  processing  time, 
^ | 

var(z)  ~ var(z)  . 


(30) 


Roughly  speaking,  the  long-term  average  may  be  considered  an  average  of  the  short-term 
average,  with  independent  samples  at  intervals  of  r.  A more  exact  treatment  is  found  in 
Appendix  B. 


SUMMARY  OF  RESULTS 


Let  us  write  the  weight  update  algorithm  for  the  ALE  in  the  following  form 
wg(k+l)  = Wg(k)  + 2/i  e(k)  Xg(k)  , (31) 

where  Xg(k)  is  defined  in  (5),  and  e(k)  is  the  error  at  time  k as  pictured  in  figure  2.  Define 
the  following  quantities 

-y  2 

i>~  = input  noise  power  = E(n£ ) 

SNR  = total  signal  power /p- 

* _ L SNR 

a 2 L 

" 1 +-^SNR 


7“  = E(n'~) 

-nv2  (l  +(1  -a*)SNR)  , (32) 

where  n and  n'  are  the  noise  processes  of  equations  (5)  and  (6),  and  SNR  is  the  input  signal 
to  noise  ratio. 


Figure  2.  Block  diagram  of  the  Adaptive  Line  Enhancer  (ALE). 
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It  then  follows  [2,3]  that  Wj  has  the  form  (6)  and 


A2  = 2v2  SNR 


E(ng2)  = 72 

B2  = A(a*)2 


where 


2iiv2  (l  + -ySNR) 


L = ALE  filter  length  . 


Let  K be  the  DFT  length  in  the  ALE  output  detector  pictured  in  figure  2 and 
assume  that  max  (2K,  K + L)  « t.  Then  a single  output  of  the  detector,  y,  with  no 
averaging  has  mean  [ 1 ] 

Jiy  = E(y) 


where 


K2L 
4 “12 


K < L 


L2K  L3 
4 “12 


K > L . 


Its  variance,  as  computed  in  Appendix  A,  is 
var(y)  = E(y2)  - 


where 


E(y2)  = 74  A2 v2  D j + 74  *>4  D2  + B4  A2  v 2 D3 

+ B4  v4  D4  + B2 72  A2  v2  D5  + B272  v4  D( 
+ 74A4D7  + B272  A4Dg  + B4  A4D9  , 


I 


and 


Dl  = 


d2  = 


d4  = 


D5  = 


K2[2KL2  - L3/3 1 , 

K > L 

K2[KL2  + K2L-K3/31  , 

K < L 

4K2L2-yKL3  + y-  , 

K > L 

2K2L2  + jK3L-^j-  , 

K.  < L 

i^2t  2 ^ , 

^-[KL2-L3/31  , 

K>  L 

!^tT[K2L-K3/31  , 

K < L 

^[KL2-L3/3]:  , 

K>L 

1 j i 2 

£[K-L-K3/31  . 

K < L 

^p[4KL2-L3] 

K > L 

[3  K2L  + KL2  - K3) 

K<  L 

(37) 


(38) 


(39) 


(40) 


(41) 


°6  = 


2K2L3-KL4  + ^L5  , 


K > L 


-^-^  + tK2L3- ^K3L2  + |lK4- |^K5  , K < I.  < 2 K 


6(3 


2K  < L 


(42) 


D7  = K4  L2/8 
Dg  = K4  L3/ 1 6 
Dq  = K4  L4/256 


(43) 

(44) 

(45) 
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Now,  suppose  the  output  is  averaged  over  a processing  time  T 


where 


N = T/max(2K,  K + L) 


Then 


varU)  = -^-j  (E(y~)  - E(r?4)) 

+ q(-jE(y2)  - My“  + 2" >) 


(46) 


(47) 


(48) 


where 


E(r?4)  *s  8*ven  ^ (26) 

E(rj4)  = 74  A4  D7  + B2  72  A4  D g + B4  A4  D9  , 
and  q is.  from  (B-8), 

lT*7(c'2T/r-|,l  ' 

Note  that  for  large  T 

T » T . 

i 

If  we  write 


T 

q ~ T 


Nn  = r 


M = N/N0 


(49) 


(50) 


(51) 


(52) 
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then 


(S3) 


which  is  (30). 


SIMULATION 

M.  Dentino  (51  provides  several  Monte  Carlo  simulations  of  the  ALE  for  a sinusoid 
in  white  Gaussian  noise  and  plots  their  ROC  curves.  Figure  3 contains  the  results  of  one  of 
these  simulations,  for  which  the  conditions  max(2K,  K + L)  « t and  r « T are  satisfied.  The 
corresponding  ROC  curve,  computed  using  the  formulae  of  this  report  under  the  assumption 
of  a Gaussian  output,  appears  in  the  same  figure.  The  results  are  in  excellent  agreement. 

Note  that  in  this  case  K = L and  t/K  ~ 72  and  T /r  ~ 8.5. 


Figure  3.  Comparison  of  computed  ROC  curve  (solid 
line)  to  Monte  Carlo  simulation  (X)  for  Pp  = 10“^, 
T = 629,760,  uv2  = 7.6  - 10~6.  and  K = L = 1024. 
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MOMENTS  RATIOS  - NOISE  ONLY 

For  noise  only,  we  have 

My  = 72  v2-  LK 

4K2L2-jKL3  + L4/3  , K>L 

E(y2)  = 74  v 4 

2 K2  L2+  jK3L-K4/3  , K<L  . 


Thus 


K » L 

K = L 
K « L . 


Since  this  ratio  is  1 for  a chi-square  distribution  of  two  degrees  of  freedom,  the  output 
distribution  is  in  general  not  chi-square  as  hypothesized  in  [ 1] . Nor  is  it  Kg  (ratio  1/3)  for 
K = L as  maintained  in  [6] . 


- 


r 


APPENDIX  A:  E(y2) 

Our  starting  point  is  equations  (5)  and  (6) 
xk_g  = xg(k)  = A cos  cu(k  - 8)  + n(k  - 8) 

E (n(8)n(m))  = v2  6gm 
wg  = wg(k)  = B cos  (w8  + <*>)  + n'g 

E(n8  nm]  = y2  68m 
E(n  n')  - 0 

where  wg(k)  is  assumed  independent  of  k for  max  (2K,  K + L)  « t (see  (B-2)). 
Note  that 

K L 

7(w  * x)  = J e'iwk  ^ wg  xk_g 
k=l  8=1 


(A- 1 ) 


= £ le-'^  wge-^k'C)  xk_g 
k 8 


K-8 


= £ e-iwS  wg  X e'iWS  *s  • 
8=1  s=l— 8 


Thus 


E(y2)  = E17(w*x)|2 
L 


^(8,-82+83-84)0; 


= E|  w8,  w8t  w83  w84 

81.82,83.84=1 

V1  v2  Y3  Y4 

• Z i ^ xsixs2xs3xs4 

s 1 = 1 —8 1 s 2= 1-8 2 s3= 1-8 3 S4-I-84 

| 

i(S|-S2+S3~S4)o; 
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( 


(A-2) 


*WKNO  not  BUMUNDT  PZI 


The  products  of  w and  x each  have  three  non-zero  types  of  terms, 
type  1 : cos  cos  cos 
type  2:  cos  cos  n 
type  3:  n n n 

All  terms  containing  an  odd  number  of  noise  terms  are  zero  since  E(n)  = E(n')  = 0.  An  even 
product  of  n’s  will  produce  a delta  function.  The  second  type  will  contain  6 = (2)  terms. 

We  shall  denote  the  product  of  a “w  term”  of  type  i with  an  “x  term”  of  type  j by  (i;  j). 


cos 

n 

n 


(6  terms) 


(A-3) 


TERMS  OF  TYPE  (w;2) 

An  x term  of  type  two  contains  the  sum 
K-8k  K-C; 


^ 5(sk-Sj)e 
sk=1-®k  sj~  * -^j 


±i(sk±sj) 


•v2k2  ^ ^ (cossnw)(cossmco)e 

sn  sm 

K-min(8j,8k) 


±i(sn±sm)w 


2 


I 


±i(sk±sk) 


K-max(Cj,?k) 


The  exponential  is  zero  only  for  (j,k)  = (1 ,2),  (2,3),  (3,4)  or  (1 , 4).  For  a non-zero  ex- 
ponential the  sum  is  of  the  order  1 , otherwise  it  is  max(K  - |Cj  --  C--I,  0)-  Thus 


(w;2) 


kzA“K‘ 


0(1),  (j,k)  = (1,3)  or  (2,  4) 

max(K  - |8j  - fikl,  0) , other 


(A-4) 


TERMS  OF  TYPE  (w3) 

The  fourth  moment  of  the  Gaussian  process  is  E(n4)  = 3 1>4.  Let  (3  be  the  number  of 
terms  for  which  s j = s2  = s3  = s4  (can  be  shown  to  be  K - max  8j  + min  8j).  Then 
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(w,3)  = £ 6(sj  -s2)  6 (s3  -s4)  v4 
sl*s3 

+ 8(S|  - s4)  8(S2  - S3)  v4 

s\*s2 

+ S 6(sj -S3)6(s2-s4)i/4e“iUJ(S3  S4^ 
sl^s2 

+ 3pv4  . 


(A-5) 


As  will  be  seen  in  the  final  results,  individual  terms  give  contributions  proportional 
to  powers  of  K and  L.  In  any  sum  such  as  (A-5),  terms  of  lower  order  may  be  neglected. 

For  example  in  (A-4),  after  summation  over  w (i.e.,  8j  and  8^.),  the  upper  term  is  of  order 

K“  whereas  the  lower  is  of  order  K“L  or  K^.  In  particular,  oscillatory  terms  such  as  the 
third  term  of  (A-5)  will  be  of  lower  order.  (The  reader  should  see  ( 1 ] for  more  informa- 
tion.) In  what  follows,  lower  order  terms  will  be  pointed  out  and  discarded  without  further 
comment. 

The  third  and  fourth  terms  of  (A-5)  are  of  lower  order.  Also,  the  effects  of  setting 
sj  * S3  and  sj  # st  in  the  first  two  may  be  neglected.  Hep~e, 

( w.3 ) = i>4  | max  (0,  (K  - |8,  - 82I)(K  - I83  - «4I)) 

+ max  (b,  (K  - |?j  - 64IXK  - |B2  - ®3I))  J • (A-6) 


TERM  (3,  2) 


There  are  three  significant  non-zero  terms  of  type  3 in  w.  Consider  first  8]  = 83) 
83  = 84,  8j  * 83.  This  gives  rise  to 


74  / ^ (2  K + 2 max  (0,  K - |8j  - C3|) ) 
'*1^3 


#,2 


which  contains  (see  (A-4))  four  contributions  from  x-terms  plus  two  contributions  from 
x which  are  of  lower  order  and  have  been  ignored.  The  term  8j  = 84,  83  = 83,  8j  # 83  is  the 

same  as  above,  whereas  the  term  = ®3>  ®2  = ®I  *®2 's 


7 


4 


"S'  i(2e, -28^)00 

Z-i  e *4 

e,#e2 


max  (0,  K — |8 1 


8?l) 


A2K2 


2 

IT  . 
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This  is  of  lower  order  and  may  be  neglected.  The  two  contributing  terms  are  identical, 
hence. 


(3 


.2)  = /a2^2--  |2(  X (2K  + 2max(0,K-|g,-83l)))j 
®1’®3 


We  note  that 


2 


e,,e 


max  (0,  K-  |8j  -C3I)  = 1 


1,K3 


I 

M3 

I 


ie,-e3l<K 


(K-|C,-C3I)  k>l 

K<L 


Also, 


, L c2  \ 

«|.*2  *2\  *1”*2  'r1  / 


= 2 


/(L  - C->)(L  - 8-)  + 1 ) (C-,  - 1)  C2> 


^4-  (L2-2Le2  + 2e?  + L-2C2) 


i3  l3  l3 

= — - — + lower  order  terms 


(A-7) 


(A-8) 


Thus, 

S (K-l«!  -C3I)  = K L2-L3/3  . 

«l-«3 

The  situation  is  somewhat  more  complicated  for  K < L.  We  first  calculate  some 
identities.  Ignoring  lower  order  terms,  we  have  for  L > K 
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1 


r 


K 


8: 


K 


~ L- 

s ..  s .♦  i s’  .♦  z r 

|Cj-Ckl<K  Cj,Ck=i  Cj=K+l  Ck=Cj-K  8k=K+l  8j=8k-K 


= k~  + i *♦  i 


K 


Cj=K+l  8k=K+l 


= K“  + K(L  - K)  + K(L  - K.)  + lower  order 


= 2K  L-  K- 


K L K 

X <Cj-cki  = I iej-ckl  + 2 ^ J <cr 


(A-9) 


8k)  + lower  order 


Vk 


|8j-«kl<K 

_k_3  + , V JL: 

3 * Z#  2 


«j=K+»  «j~ek=o 


+ lower  order 


arK+ 1 


= -^'  + K2L-K3  = K2L-|K3 


Substitution  of  these  expressions  in  (A-7)  yields 


(A- 10) 


(3,2)  = 74  A2i>2K2 


2KL2- L3/3 


K>L 


(A- 1 1) 


K L“  + K‘L-  KJ/3  K < L 


TERM  (3,3) 

As  in  the  previous  case  we  consider  the  separate  W terms.  For  8 1 = 83,  83  = 

8)  =£  83  we  have 

yM  N k2+  (max(0,K-  |8, -83l))2|  . 

«1^3 

For  8,  = 84;  8t  = 83;  8,  * 83,  the  result  is  the  same.  For  8,  = 83;  83  = 84;  8,  & 83,  a factor 
2iui(8i-8-i) 

of  e “ enters,  and  hence  it  is  of  lower  order. 
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Let  us  evaluate  2 |Bj  - B3I“.  For  K > L,  we  have 

V |«1-B3I2=  ^ (C,-fi3)2 

«l-«3  M3 


= 7 «?-2 


*1*3 


J-2C183  + S 


= 2—  -2l  — 
1 3 ■* 


+ lower  order 


= L4ijU4 
V 3 2 1 6 


(A- 12) 


and  for  K < L 


K L K 

\ ' m n |2  i 1 \ ^ 


y |Cj  _ Ckl2  = ^ |Bj-Bkl2+2  y y (Bj-ek):  + lower  order 


l«r«kl<K 


cjck 


Bj=K+l  Bj-Bk=0 


i/4  ^ 

— + 2(L  - K)(K^/3)  + lower  order 


_Kl  + 2 3,  K^2  =2k3l_K2 

6 3KL  3 3 K 2 


;4-> 


( A- 13) 


Thus 


(3.3)=  2 y*vA 


K2  + 


1 (K2  - 2K1C,-C31  + ICj -C3I2) 

|BrB3l<K 


4K2L2-|kL3+^-  , 

K>L 

2K2L2+jK3L-^y  , 

K<L 

(A- 14) 
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TERM  (1,2) 

The  type  1 terms  contain  four  cosines.  The  summation  over  two  of  them  gives 

B2  Z(cos  emGj)(cos  2nco)  cos  (2m±2n)a>  = B“  L2/4. 

From  (A-4),  we  see  that  only  four  of  the  type  two  terms  are  significant.  Each  of  these  gives 
a contribution  of  (from  (A-4)) 


B2  L2 


1 


(cos  to  2j)(cos  to  2k)(cos  co(8j  - 2k)) 


.2  .-2  2 
A h.  v 


• max  (0,  K - |2j  - 2kl) 

We  now  use  the  identity 

1 . cos  2(a±b)  + cos  2a  + cos  2b 

(cos  a)(cos  (a±b))(cos  b)  = ^ + ^ — — 4 

All  terms  but  the  1/4  will  be  of  lower  order.  Thus  the  above  becomes 

b2aVk2l2  I V m,nI,  l0  p ,, 

I ^ ^ . m3x  (0,  K.  |Vj  ) 

2i,2k 

which  with  ( A-9)  and  (A- 1 0)  yields 


(1,2)  = 


B4  A2  v2  K2  L2 
16 


KL2  - L3/3  , K>  L 

K2L-K3/3  , K<L 


(A-15) 


(A-16) 


TERM  (1,3) 

Proceeding  as  for  term  (1,2)  and  using  (A-6),  we  get  two  terms  of  the 


form 


B4i>4 


y (cos  2jo;)(cos  22<o)(cos  (2j  max(0,  K-  |2j  - 22I) 

^ (cos  2|oc)(cos  22to)sin  (2|-22>u>  max(0,  K-  |2  j — C2I) 
el’S2 


1 2 
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since 


cos  u>(8  j - ?2  + ~ ^4)  = cos  (2 1 ~ 82)<o  cos  (®3  “ ^4)°° 

- sin  (8  j - 82)cj  s'n  (83  - 84)^  • 

Because  of  the  sine,  the  second  term  is  of  lower  order.  Also,  according  to  (15),  the  first 
term  equals  1/4  plus  terms  of  lower  order.  Thus 

2 

(1,3)  = 2 BV  (j  ^ max  (0,  K — |C , — C2I )) 

Cj,c2 

„ . (KL2-L3/3)2  K>1 

= bV  (/ 

8 (K2L-K3/3)2  K<L 


TERM  (2,2) 

As  a consequence  of  ( A-4),  we  are  only  concerned  with  the  sets  of  indices  j.k  = 1 ,2  or 
1,4  or  2.3  or  3,4.  There  will  be  four  terms  in  which  the  indices  of  the  two  cosines  in  w 
coincide  with  j and  k,  four  terms  in  which  they  are  disjoint,  and  sixteen  terms  in  which  one 
index  coincides  and  the  other  does  not.  We  note  that 


I 


2 1 2 

y = L-r 


e e 

xnr  n 


Thus  (2,2)  will  contain 


(cos  co8j)(cos  Cfcto)  cos  max  (0.  K-  IBj-BjJ) ^ 


+ 4 ^ (cosCmoj)(cosCnco)  cos(Cm-8n)cj  6(Cj-C|c)max(O.K-|Cj-8|cl) 

p.  e.  c ? 

*j>  k’ m n 


A2kV 


t2b2 


+ 16  ^ (cos  8jto)(cos  8mw)  cos  (8j  ± 8m)co  max  (0.  K-  Kj— 8^1)  6(8n—  8^)  cos(8n  ± 8^)00 

®j’®k'®m’®n 

A2K2l'2  2n) 

— y * ■ 
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From  (A-8)  and  (A-l  5),  the  first  term  is 


2a2  1 K~L 
B-7“A-v“  -j- 


The  second  term  is 


KL2  - L3/3 
K2L-K3/3 


B272aVk2  kl3 


The  third  term  yields 


L<  K 
K < L 


^ (cos2Cjo;)max(0,K-  |Cj— l)5(Cn— C^)  cos(Cn  ± 

VVn 


A2  K2  ■}  ? 

— „-7-  B 


V ->  a2k2  ■>  ■>  ■ 

= 8L  / (cos-CjCo)max(O, K-  |Cj— Cj^l ) cos (Cj^ ± y*- B‘ 
6j’6k 

Since  those  terms  containing  a plus  sign  are  of  lower  order,  four  terms  remain: 

4L  ^ (cos2  Cjco)  max  (0,  K - |Cj  - C^l)  y2  72  B2  . 

Vk 

Also  cos2(Cjce)  = 1/2  + (cos  28jio)/2);  so  that  neglecting  lower  order  terms,  we  have 


d2„2  a2  2 K‘L 
B 7 A v — — 


KL2  - L3/3 
K2L-K3/3 


K>L 
K < L 


Equations  (A-l  8),  (A-l  9)  and  (A-20)  yield 


(2,2)  = B~yzAli> 


2^2  * 2., 2 Kit 


4 KL2  - L3 
3 K2L  + KL2  - K3 


K > L 
K < L 


TERM  (2,3) 


There  will  be  four  terms  in  which  the  cosine  terms  of  w match  at  least  one  pair 
of  indices  in  (A-6)  and  two  terms  in  which  they  do  not.  The  first  four  give 


4B27V  (cos  C|w)(cos  ^cj)  cos(Bj  - 82^ 

® 1 »®2*®3 

• | K max  (0,  K-  |erB2l)  + max(0,  K-  |«rC3l)max(0,K-  |82-«3l)| 
The  usual  identity  (A-l  5)  yields 


B27V  ^ LK  max  (0,  K-  |C j — C2I) 

® 1 

+ ^ max  (0,  K-|8r«3l)  max  (0.K- IB2-B3I)  . (A-22) 

® 1 »®2’®3 

The  other  terms  are 

2 B272y2  ^ (cos  B|U))(cos  cos  (B|  + 63)0;  cos  (2B2<*>) 

C 1 ,C2’C3 

• | max  (0,  K - |8|  - 82I)  max  (0,  K - IC3— C2I)  + max(0,  K — |C  | — C2I) 

• max  (0,  K - IC2  — C3I)  J 

which  is  of  lower  order  because  of  the  cos  (2^2^)  factor. 


In  order  to  complete  the  evaluation  of  (2,3)  we  need  some  additional  summation 
identities.  For  K > L,  with  lower  order  terms  omitted. 


^ l«l-e3U«2-*3l*  X £ (83-82)l|l|-«3l 

«|.«2-«3  *l-e3  L*2=i 


L 

+ £ (82-63)18,-831 

«2=e3 

= 1 IC1  -c3l(^r)  (l2-2LC3+  2C^) 

8,, 63 

= S i(L2-2LV2*“)2 

e3 


= | X (L4+8L2fi3  + 4?t~4L3c3-8LC3) 

C3 

= + (l5  + |l5  + jL5-2L5-2L5) 

*^l5  ■ (A-23» 
(Note  that  these  summations  are  easily  performed  using  the  formula 
b 

^ 8n  = (bn+  * - an+  * ) + lower  order  terms  , 

8=  a 

which  may  be  proved  by  considering  integral  approximations.) 

For  K < L,  let  18,  - 83I  = s and  18 2 - 83I  = m- 

Then,  since  the  factors  max  (0, 16,- Cj|)  restrict  the  summations  to  |s|<K,  |m|<K,  for 
8,  > 83  s ranges  from  0 to  min  (K,  L - 63);  and  for  6,  <,  83  it  ranges  from  0 to 
min  (K,  63-!).  Thus 
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r 


L /min(K,L-e3)  min(M3-l)V  /min(K,L-83)  min(K,C3-l)> 

2=2  2 * \ ){  l + s 

s<K  Ci=l  \ s=0  s=0  / \ 


m=0 


m=0 


m<K 


L /min(K,L-C3)  min(K,83-l)' 

= 2(2  ♦ 2 

S3=l  \ s=0  s=0 


(A-24) 


We  now  must  consider  the  two  “sub-cases.”  2K.  < L and  K < L < 2K.  For  2K<L, 
we  have  from  (A-24) 

V 1/  \ - 


K / K 

*r*N 

2 L-K+l 

s = 

+ V 

(♦  2 

Z 

s<K 

\ i— • 

C3=l  \s=0 

si ) 

e3=K+l 

m<K 

L 

r-»  i 

/L& 

+ - 
83=L-K 

1 ♦ 
V s=0 

2 ) 

s=0  / 

K 

L-K 

V-' 

= Z (K  + «3)2  + 

Z <2K)2 

tO 

u> 

II 

*3 

-K+l 

c3=l-k 


K 


= K2+  2K83  + 8“  + 4K2(L-2K) 


e3=1 


+ £ (L-fi3)2  + 2K(L-fi3)  + K: 

c3=l-k 


= K3+K3+^-  + 4K2L-8K3+^y+K3  + K3 


= 4K2L-  ^K3 
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(A-25) 


• -- 


2K3L-||k4 


(A-26) 


and 


2 


m<K 


t l (K^  + £ («3>+£) 

C3=K+!  e3=L-K 


-2K) 


2 

i=L-K  V 


= JT  + iT  + To+K4L-2K5  + To  + i7+iT=K4L-TfK5  • (A’27) 
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Thus  for  2K  < L,  from  (A-25),  (A-26),  and  (A-27), 


^ (K-sKK-m)  = 4K4L-yK5-4K4L+^K5  + K4L-  j|  K5 

s<K 

m<K 


-k-4,  12  k5 

K l_30K 


Similar  techniques  are  used  for  K < L < 2K: 


m<K 


L-K 


K L 

(K  + C3)2+  ^ L2+2  (L  + K-C3)2 

c3=l-k  c3=k 


=-^(L3  - K3)  + L2(2K  - L)  +-j(L3  - K3) 
= 2L2K-L3/3-~K3  . 


Also, 


2*= 


s<K 

m<K 


(K  + C3)  + 


K 

2 


2 2 /L 


c3=l-k 


+ 


(L  + K-e3) 


(A-28) 


(A-29) 
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r /k3  K2  Ke3  \ L / 3 3 3 

= 2 (-V+V  «3+^+t/ +t\k3-(L-K)  '(L_K)3+K 

«3=1 

V (L-«3)3  (L-C3)2  (L-6,)K2  k3 

+ Z 2+K2  + 2 + 2 ’ 


c3=k 


which  after  more  calculation. 


Finally, 


= -LK3-^  +^L2K2-3KL3  + L4/3  . 


L-K  1 2 \ 2 K / , „2\2 

s - 1 k+{)  * x 4) 

✓-i/  n , 1 ' ' n -1  1/  ' ' 


s<K  C,=  l 

m<K 


6 3=  L-K 


(L-C3)2  i/2  \2 

+ 7 i — +v 


«3=K 


which  after  some  computation  yields. 


I 

s<K 

m<K 


sm,tK4.8j^.2K:L3+iK3L:  + iL4K.ti 


Combining  (A-29),  (A-30),  and  (A-31),  we  have  for  K < L < 2K, 


s<K 

m<K 


(K-sXK-m)  - ^ ♦ |K2L3  - £ K3L2 


+ 3LK4--j^K5 


3) 


(A-30) 


(A-31) 


(A-32) 
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We  now  substitute  (A-9),  (A- 10),  (A-23),  (A-28),  and  (A-32)  into  (A-22)  to  get 


( 2K2L3-KL4+g^LS  , 


(2,3)  = B272i>4 


K>L 


KL^  _3  ^2l3_  L2  k3.  2 + 8.1  — k5 

60  A K L + ,LK  -tnK  , 


L2K3+|k4L- jjK5  , 


3"~  ~ 10 
2K  > L > K 
L>2K  . 


(A-33) 


TERMS  (1,1) + (2,1) + (3,1) 

We  note  that  since  the  cosine  function  is  periodic,  the  summation  in  (A-2)  for  x of 
type  1 may  be  rewritten  (see  (22))  so  that  the  product  formula  for  the  convolution  holds. 
Thus, 


(1,1)  + (2,1)  + (3,1)  = E 17(w)!4  |7( A cos  wk)|4  . 

The  second  factor  is  ( AK/2)4.  The  first  is  given  by  (25).  Consequently 


( 1 ,1)  + ( 2.1 ) + (3,1 ) = 


AV/2LV+L3„VtLV 


(A-34) 


RESULT 

The  final  result  is  given  by 

E(y2)  = (1.1)  + (2.1)  + (3.1)  + ( 1,2)  + (2,2)  + (3,2)  + (1,3)  + (2,3)  + (3,3)  , 
all  of  which  have  been  computed  above. 
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APPENDIX  B:  TIME  AVERAGING  THE  WEIGHT  VECTOR 

At  low  signal-to-noise  ratios,  the  eigenvalues  of  the  correlation  matrix  of  the  input 
to  the  ALE  will  be  approximately  equal,  and  the  system  of  linear  difference  equations  (31) 
will  exhibit  a single  time  constant  r given  by  (33)  [1,7].  Then,  with  equation  (6)  as  an 
initial  condition 


wc(t)  = n'g(0)  e-t/T+  ct 


(B-l) 


where  ct  is  independent  of  n'g(O)  since  it  depends  only  on  inputs  after  t = 0.  Since  n'g(t) 
differs  from  Wg(t)  only  by  a non-random  quantity  (6),  and  since  E n^( t)  = 0, 


E(n'g(0)  n'g(t))  = E (n“(0)J  e~t,T  = y2  e"t/r  t > 0 


(B-2) 


This  dependence  is  also  mentioned  in  (6] . 

Let  us  now  consider  the  time  average  of  w. 


1 

d t 


f Wg(t) 


dt  . 


The  variance  of  q is  given  by 


var(q')  = — E 
T- 


T T 

f J ng(t)  ng(s)  dsdt 
0 0 


2 T T 

= JT  e-lt-sl/T  ds  dt 

T“  r\  n 


0 0 


y2  [T  + T(e-Jlr-  D]  . 
T2 


(B-3) 


For  large  T 


lim  var(q')  = 

T-*«  1 


(B-4) 


Note  that  the  variable  v appearing  in  equation  (7)  is  homogeneous  in  w f of  order  2 
(i.e.,  depends  on  wjw^),  and  thus  by  factoring  out  (e-1^*"  we  have 


E(v(t)v(0))  = E(v2(0))  e-2t/T 


(B-5) 
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